No-Φ-Regret: A Connection between Computational Learning Theory and Game Theory

This paper explores a fundamental connection between computational learning theory and game theory through a property we call no-Φ-regret. Given a set of transformations Φ (i.e., mappings from actions to actions), a learning algorithm is said to exhibit no Φ-regret if an agent experiences no regret for playing the actions the algorithm prescribes, rather than playing the transformed actions prescribed by any of the elements of Φ. The existence of no-Φ-regret learning algorithms is established, for all finite Φ. Analogously, a class of game-theoretic equilibria, called ~ Φ-equilibria, for ~ Φ = (Φi)1≤i≤n, is defined (here n is the number of agents/players). The main contribution of this paper is to show that that the empirical distribution of play of no-Φi-regret algorithms converges to the set of ~ Φ-equilibria. The well-known result that the empirical distribution of play of nointernal-regret learning converges to the set of correlated equilibria follows as an immediate corollary of this general theorem. In addition to providing a sufficient condition, a necessary condition for convergence to the set of ~ Φ-equilibria is also derived. This work was originally motivated by an attempt to design a no-regret learning scheme that would converge to a tighter solution concept than the set of correlated equilibria. However, it is argued that the strongest form of no-Φ-regret learning is no-internal-regret learning. Hence, the tightest game-theoretic solution concept to which any no-Φ-regret algorithm converges is correlated equilibrium. In particular, Nash equilibrium is not a necessary outcome of learning via any no-Φ-regret algorithms.